Prioritizing instructions based on type

ABSTRACT

Methods and reservation stations for selecting instructions to issue to a functional unit of an out-of-order processor. The method includes classifying each instruction into one of a number of categories based on the type of instruction. Once classified an instruction is stored in an instruction queue corresponding to the category in which it was classified. Instructions are then selected from one or more of the instruction queues to issue to the functional unit based on a relative priority of the plurality of types of instructions. This allows certain types of instructions (e.g. control transfer instructions, flag setting instructions and/or address generation instructions) to be prioritized over other types of instructions even if they are younger.

BACKGROUND

Superscalar processors are designed to issue more than one instructionper clock cycle. Generally these processors comprise one or morereservation stations that receive and store instructions until they areready to be issued to the out-of-order pipelines for execution. Aninstruction is said to be ready for issue when all of its input operandsare available.

Each clock cycle the reservation station selects a number of readyinstructions (e.g. one per out-of-order pipeline) and issues them to theout-of-order pipelines for execution. However, in any given clock cyclethere can be a number of instructions that are ready for issue.Accordingly, there must be some mechanism for selecting readyinstructions to issue to the out-of-order pipelines.

Typically the highest priority is given to the oldest ready instructionsand the lowest priority is given to the youngest ready instructions.Issuing the oldest instructions as soon as possible ensures that there-order buffer does not fill up and stall the dispatching ofinstructions. However, this policy alone does not ensure optimumprocessor performance.

The embodiments described below are not limited to implementations whichsolve any or all of the disadvantages of known processors.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Described herein are methods and reservation stations for selectinginstructions to issue to a functional unit of an out-of-order processor.The method includes classifying each instruction into one of a number ofcategories based on the type of instruction. Once classified aninstruction is stored in an instruction queue corresponding to thecategory in which it was classified. Instructions are then selected fromone or more of the instruction queues to issue to the functional unitbased on a relative priority of the types of instructions. This allowscertain types of instructions (e.g. control transfer instructions, flagsetting instructions and/or address generation instructions) to beprioritized over other types of instructions even if they are younger.

A first aspect provides a method of selecting instructions to issue to afunctional unit in an out-of-order processor, the method comprising:receiving a plurality of instructions at a reservation station, eachinstruction being one of a plurality of types of instructions and beingclassified into one of a plurality of categories based on the type ofinstruction; storing, at the reservation station, each instruction inone of a plurality of instruction queues, each instruction queue beingassociated with one of the plurality of categories; and selecting, atthe reservation station, instructions from one or more of theinstruction queues to issue to the functional unit based on a relativepriority of the plurality of types of instructions.

A second aspect provides a reservation station to select instructions toissue to a functional unit in an out-of-order processor, the reservationstation comprising: an instruction classifier configured to receive aplurality of instructions, each instruction being one of a plurality ofinstruction types and being classified (or configured to classify) intoone of a plurality of categories based on the type of instruction; aplurality of instruction queues in communication with the instructionclassifier, each instruction queue configured to receive instructionsfrom the instruction classifier that have been classified into aparticular category and store the received instructions; and aninstruction scheduler in communication with each of the plurality ofinstruction queues, the instruction scheduler configured to selectinstructions from one or more of the instruction queues to issue to thefunctional unit based on a relative priority of the plurality of typesof instructions.

A third aspect provides a processor comprising the reservation stationdescribed herein.

Further aspects provide a single-threaded out-of-order processor assubstantially described with reference to FIG. 1, a reservation stationas substantially described with reference to FIG. 2, a method ofselecting instructions to issue to a functional unit as substantiallydescribed with reference to FIG. 4 and a multi-threaded out-of-orderprocessor as substantially described with reference to FIG. 5.

The methods described herein may be performed by a computer configuredwith software in machine readable form stored on a tangible storagemedium e.g. in the form of a computer program comprising computerreadable program code for configuring a computer to perform theconstituent portions of described methods or in the form of a computerprogram comprising computer program code means adapted to perform allthe steps of any of the methods described herein when the program is runon a computer and where the computer program may be embodied on acomputer readable storage medium. Examples of tangible (ornon-transitory) storage media include disks, thumb drives, memory cardsetc and do not include propagated signals. The software can be suitablefor execution on a parallel processor or a serial processor such thatthe method steps may be carried out in any suitable order, orsimultaneously.

The hardware components described herein may be generated by anon-transitory computer readable storage medium having encoded thereoncomputer readable program code.

This acknowledges that firmware and software can be separately used andvaluable. It is intended to encompass software, which runs on orcontrols “dumb” or standard hardware, to carry out the desiredfunctions. It is also intended to encompass software which “describes”or defines the configuration of hardware, such as HDL (hardwaredescription language) software, as is used for designing silicon chips,or for configuring universal programmable chips, to carry out desiredfunctions.

The preferred features may be combined as appropriate, as would beapparent to a skilled person, and may be combined with any of theaspects of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be described, by way of example, withreference to the following drawings, in which:

FIG. 1 is a schematic diagram of an example single-threaded out-of-orderprocessor;

FIG. 2 is a schematic diagram of an example reservation station andassociated functional unit of FIG. 1;

FIG. 3 is a schematic diagram of an example instruction queue of FIG. 2;

FIG. 4 is flowchart of an example method for selecting instructions toissue to a functional unit;

FIG. 5 is a schematic diagram of an example multi-threaded out-of-orderprocessor;

FIG. 6 is a schematic diagram showing example queues for amulti-threaded out-of-order processor; and

FIG. 7 is a schematic diagram showing example interleaved queues for amulti-threaded out-of-order processor.

Common reference numerals are used throughout the figures to indicatesimilar features.

DETAILED DESCRIPTION

Embodiments of the present invention are described below by way ofexample only. These examples represent the best ways of putting theinvention into practice that are currently known to the Applicantalthough they are not the only ways in which this could be achieved. Thedescription sets forth the functions of the example and the sequence ofsteps for constructing and operating the example. However, the same orequivalent functions and sequences may be accomplished by differentexamples.

It has been identified that executing certain types of instructions(e.g. control transfer instructions, flag setting instructions andaddress generation instructions) as soon as possible, even if they areyounger than other ready instructions, may significantly improve theperformance of a processor. This is because once these types ofinstructions have been issued they free up other resources which can beused by other instructions.

Accordingly, embodiments described herein relate to methods, reservationstations and processors for prioritizing certain types of instructionseven if they are younger than other ready instructions. In particular,in the embodiments described herein instructions are sorted in areservation station into one of a plurality of instruction queues basedon the type of instruction. The reservation station then selectsinstructions for execution from the instruction queues based on arelative priority of the plurality of types of instructions. This allowscertain types of instructions to be prioritized over other instructionseven if they are younger than other ready instructions. Selectinginstructions in this manner ensures that higher priority instructionsare selected for issue over lower priority instructions, therebyguaranteeing the higher priority instructions functional unit resources(e.g. pipelines) and increasing the performance of the processor.

Furthermore, selecting instructions in this manner does not introduceany additional delay in the processor. In particular, even aftersegregating different types of instructions into separate instructionqueues the selection and issuing processes can be completed in a singlecycle.

Reference is now made to FIG. 1 which illustrates a single-threadedout-of order processor 100 wherein certain types of instructions aregiven priority over other instructions.

The processor 100 comprises a fetch stage 102 configured to fetchinstructions from a program (in program order) as indicated by a programcounter (PC) and a decode and renaming stage 104 arranged to interpretthe instructions and perform register renaming. As is known to those ofskill in the art register renaming enables more instructions to beexecuted out-of-order, by removing certain dependencies, such aswrite-after-read (WAR) dependencies.

After an instruction passes through the decode and renaming stage 104,it is (a) inserted into a re-order buffer 106 (ROB); and (b) dispatchedto one of the reservation stations 108 a, 108 b or 108 c.

The re-order buffer 106 is a buffer that enables the instructions to beexecuted out-of-order, but committed in-order. The re-order buffer 106holds the instructions that are inserted into it in program order, butthe instructions within the ROB 106 can be executed out of sequence bythe functional units 110 a, 110 b and 110 c. Instructions are outputfrom the re-order buffer 106 to a commit stage 112 in program orderafter they have been executed by the functional units 110 a, 110 b, and110 c. The commit stage 112 then commits the results of the instructionto the register/memory.

In some examples, the re-order buffer 106 can be formed as a circularbuffer having a head pointer pointing to the oldest instruction in theROB 106 (the head instruction), and a tail pointer pointing to theyoungest instruction in the ROB 106 (the tail instruction). In theseexamples, the head instruction is output from the head of the ROB 106when that instruction has been executed by the functional units 110 a,110 b, and 110 c, and the head pointer is incremented to point to thenext instruction in the ROB 106.

Each reservation station 108 a, 108 b and 108 c receives instructionsfrom the decode and renaming stage 104 and issues the instructions to acorresponding functional unit 110 a, 110 b or 110 c for execution.Although the processor 100 of FIG. 1 comprises three reservationstations 108 a, 108 b and 108 c other processors may have more or fewerreservation stations.

Furthermore, although FIG. 1 shows a one-to-one relationship betweenreservation stations 108 a, 108 b and 108 c and functional units 110 a,110 b and 110 c respectively, in other cases there may be a one-to-manyrelationship between the reservation stations 108 a, 108 b and 108 c andthe functional units 110 a, 110 b and 110 c. For example, in some casesat least one of the reservation stations 108 a, 108 b or 108 c may beconfigured to provide instructions to a plurality of functional units110 a, 110 b and/or 110 c. In other cases, there may be only a singlereservation station 108 a, 108 b or 108 c that is configured to provideinstructions to all of the functional units 110 a, 110 b and 110 c.

In the embodiments described herein each reservation station 108 a, 108b and 108 c is configured to sort the instructions received from thedecode and renaming stage 104 into a plurality of instruction queuesbased on the type of instruction. Each clock cycle the reservationstation 108 a, 108 b or 108 c attempts to select a number ofinstructions from the instruction queues to issue to the correspondingfunctional unit 110 a, 110 b or 110 c for execution. In some cases thereservation stations 108 a, 108 b or 108 c are configured to selectinstructions from the instruction queues to prioritize certain types ofinstructions over others.

Each functional unit 110 a, 110 b, and 110 c is responsible forexecuting instructions and may comprise one or more functional unitpipelines. The functional units 110 a, 110 b, and 110 c may beconfigured to execute specific types of operations or they may begeneral purpose functional units. For example one or more functionalunits 110 a, 110 b and 110 c may be a load/store unit, an integer unit,a floating point unit (FPU), a digital signal processing (DSP)/singleinstruction multiple data (SIMD) unit, or a multiply accumulate (MAC)unit. A load/store unit performs load and store memory operations, aninteger unit performs integer operations, an FPU executes floating pointoperations, a DSP/SIMD unit has multiple processing elements thatperform the same operation on multiple data points simultaneously, and aMAC unit computes the product of two numbers and adds that product to anaccumulator. The functional units 110 a, 110 b and 110 c and thepipelines therein may have different lengths and/or complexities. Forexample, a FPU pipeline is typically longer than an integer executionpipeline because it is generally performing more complicated operations.

An example reservation station 108 and associated functional unit 110will be described with reference to FIG. 2.

The processor 100 also comprises a branch predictor 114, which isconfigured to predict which direction the program flow will take in thecase of instructions known to cause possible flow changes, such asbranch instructions. Branch prediction is useful as it enablesinstructions to be speculatively executed by the processor 100 beforethe outcome of the branch instruction is known. The branch predictor 114may be in communication with the fetch stage 102, decode and renamingstage 104, and/or the functional units 110 a, 110 b and 110 c. Forexample, the fetch stage 102 may use information from the branchpredictor 114 to determine which instruction to fetch next; the decodeand renaming stage 104 may provide information to the branch predictor114 indicating which instructions are branch instructions (in othercases the instructions may be decoded in the fetch stage 102); and thefunctional units 110 a, 110 b and 110 c may provide the branch predictor114 with information indicating updates to the program counter.

When the branch predictor 114 predicts the program flow accurately, thisimproves performance of the processor 100. However, if the branchpredictor 114 does not correctly predict the branch direction, then amis-prediction occurs which needs to be corrected before the program cancontinue. To correct a mis-prediction, the speculative instructions sentto the ROB 106 are abandoned, and the fetch stage 102 starts fetchinginstructions from the correct program branch.

Reference is now made to FIG. 2, which illustrates an examplereservation station 108 of FIG. 1 and an associated functional unit 110.As described above, the reservation station 108 issues instructions tothe functional unit 110 for execution in a manner that prioritizescertain types of instructions over other types of instructions.

The reservation station 110 comprises an instruction classifier 202 forcategorizing the instructions received from the decode and renamingstage 104 into one of a plurality of categories based on the type ofinstruction; a plurality of instruction queues 204-210 for storinginstructions in a specific category; and an instruction scheduler 212for selecting ready instructions from the instruction queues 204-210 andissuing them to the functional unit 110 for execution.

As briefly described above, the instruction classifier 202 receivesinstructions from the decode and renaming stage 104; classifies eachinstruction into one of a plurality of categories based on the type ofinstruction; and sends each classified instruction to the instructionqueue 204-210 corresponding to the assigned category. In some cases thedecode and renaming stage 104 uses the operation code (“opcode”) of theinstruction to determine the type of instruction. The decode andrenaming stage 104 then provides an indication of the type ofinstruction in the data provided to the reservation station 108. Inother cases the decode and renaming stage 104 provides the opcodes tothe reservations station 108 and the instruction classifier 202 uses theopcodes to identify the type of instruction.

In some cases the instruction classifier 202 uses the informationreceived from the decode and renaming stage 104 (e.g. indicator oropcode) to identify at least one of the following types of instructions:control transfer instructions, address generation instructions, and flagsetting instructions. Control transfer instructions are instructions,such as branch instructions, that modify the program counter (PC) toalter a program's control flow. Address generation instructions areinstructions that resolve the memory address of load and storeinstructions. It is only after execution of such instructions that aload or store operation can be successfully performed. Flag settinginstructions are instructions that modify the condition flags stored ina flag register that indicate the status of the processor.

In other cases the instruction classifier 202 may use the informationreceived from the decode and renaming stage 104 (e.g. indicator oropcode) to identify more, fewer or other types of instructions.

In some cases there may be one category for each type of instructionidentified (e.g. a category for control transfer instructions, acategory for address generation instructions, and a category for flagsetting instructions) and one general category for all otherinstructions. In other cases there may be one category for one or moretypes of identified instructions (e.g. a category for control transferinstructions, address generation instructions, and flag settinginstructions; or a category for control transfer instructions and acategory for address generation and flag setting instructions) and onegeneral category for all other instructions.

Although it has been described herein that the instruction classifier202 performs the classification of the instructions, in other examplesthe classification may be performed by another component of theprocessor 100, such as the decode and renaming stage 104. In theseexamples, in addition to receiving the instruction itself from thedecode and renaming stage 104 the instruction classifier 202 may alsoreceive information indicating which category the instruction has beenclassified into. The instruction classifier 202 then sends the receivedinstructions to the appropriate instruction queue based on theclassification information received from the decode and renaming stage104.

Each instruction queue 204-210 receives instructions from theinstruction classifier 202 for a particular category and stores eachinstruction in an available entry or slot. The instructions may bestored in the instruction queues in the order in which they are receivedfrom the instruction classifier 202. Since the instruction classifier202 typically receives and processes the instructions from the decodeand renaming stage 104 in program order, the instructions may be storedin the instruction queues 204-210 in program order.

There is typically one instruction queue 204-210 for each category. Inthe example reservation station 108 of FIG. 4 there are four categoriesof instructions (e.g. one for each type of instruction identified above)thus there are four instruction queues 204-210. For example, the firstinstruction queue 204 (instruction queue 0) may be for control transferinstructions, the second instruction queue 206 (instruction queue 1) maybe for address generation instructions, the third instruction queue 208(instruction queue 2) may be for flag setting instructions, and thefourth instruction queue 210 (instruction queue 3) may be for all otherinstructions. However, in other examples, the reservation station 108may have more or fewer instruction queues.

An instruction waits in the instruction queue until its input operandvalues are available. In particular, the reservation station 108 maymonitor the output of the functional unit(s) 110 to find matches formissing operands (operands with a ready bit set to 0) for eachinstruction in the instruction queues. Once the reservation station 108finds a match for a missing operand it sets the ready bit for thatoperand to 1. In some cases, the reservation station 108 also copies theoperand from the functional unit 110 output to the instruction queue. Inother cases, the operand value is read from the appropriate registerfile after the instruction has been issued to the functional unit 110.For example, the first stage of the pipelines 214-220 may be configuredto read the operand value from the appropriate register file. This mayreduce the number of connections between the functional unit(s) 110 andthe reservation station 108 over the cases where the operand values arecopied from the functional unit 110 output.

Once all of an instruction's operand values are available theinstruction is said to be ready for execution and a ready bit for thatinstruction may be set indicating that the instruction is ready to beissued to the functional unit 110. An instruction's operand values maybe available before the operand values for earlier, older instructionsallowing it to leave the instruction queue before those earlier, olderinstructions.

An example instruction queue 204 will be described with reference toFIG. 3.

The exemplary reservation station 108 of FIG. 2 also comprises aninstruction scheduler 212 for issuing ready instructions from theinstruction queues 204-210 to the functional unit 110 for execution.

The functional unit 110 comprises one or more like pipelines 214-220 forexecuting instructions. The exemplary functional unit 110 of FIG. 2comprises four pipelines 214-220 numbered 0 to 3; however, in otherexamples the functional unit 110 may comprise more or fewer pipelines.

Each clock cycle the instruction scheduler 212 attempts to provide eachpipeline 214-220 with an instruction from one of the instruction queues204-210. For example, where the functional unit 110 comprises fourpipelines 214-220 as shown in FIG. 2, the instruction scheduler 212attempts to select four ready instructions from the instruction queues204-210 and issue one of the selected instructions to each pipeline214-220.

In many out-of-order processors the reservation station comprises only asingle instruction queue and the oldest ready instructions in the queueare selected for issue to the pipelines 214-220. Issuing the oldestinstructions in the queue as soon as possible ensures that the re-orderbuffer 106 does not fill up and stall the dispatching of instructions bythe decode and renaming stage 104. However, this does not ensure optimumperformance of the processor. In particular, it has been identified thatexecuting certain types of instructions (e.g. control transferinstructions, flag setting instructions and address generationinstructions) as soon as possible, even if they are younger than otherready instructions may significantly improve the performance of theprocessor. This is because once these instructions have been issued theyfree up other resources which can be used by other instructions.

In particular it has been observed that mis-speculated (i.e. incorrectlypredicted) control transfer instructions have a costly recovery overheadand therefore executing control transfer instructions as soon aspossible, even if they are younger that other ready instructions maylead to significant performance improvement. Similar observations havebeen made for flag setting and address generation instructions.

Accordingly, the instruction scheduler 212 of FIG. 2 is configured toselect ready instructions from the instruction queues 204-210 in amanner that prioritizes certain types of instructions even if they areyounger that other ready instructions. In some cases, the instructionqueues 204-210 are prioritized from highest priority to lowest priorityand the instruction scheduler 212 selects ready instructions from theinstruction queues in order of priority. For example, if the instructionqueues 204-210 are prioritized in the following order from highestpriority to lowest priority—instruction queue 0, instruction queue 1,instruction queue 2 and instruction queue 3—the instruction scheduler212 may first select ready instructions from instruction queue 0, theninstruction queue 1, then instruction queue 2, then instruction queue 3.

In some cases the priority of the instruction queues 204-210 may bepredetermined or fixed, and in other cases the priority may bedynamically adjusted.

In particular, in some cases, the priority of the instruction queues204-210 may be determined by offline-testing and/or performance analysisand then predetermined or fixed in the processor. For example, it isknown to those of skill in the art that irrespective of carefullyselecting an instruction set architecture (ISA) and micro architecture,performance bottlenecks may be caused by certain instructions (e.g.control transfer instructions). By performing a detailed analysis of theimpact of the chosen micro architecture on the performance of benchmarkapplications of interest, instructions may be prioritized as seen fit ona case-by-case basis. In some cases, the relative priorities of certaininstructions may be determined by examining several parametersindicating the status of the processor at any given time.

Detailed analysis has shown that in some cases optimum performance canbe achieved if the instruction queue associated with control transferinstructions is given the highest priority, the instruction queueassociated with flag setting instructions is given the second highestpriority, the instruction queue associated with address generationinstructions is given the third highest priority, and the instructionqueue associated with all other instructions is given the lowestpriority.

In other cases the processor may comprise an additional component, suchas the Automatic MIPS allocator (AMA), which monitors the overallperformance of the processor and generates appropriate control signalsto the instruction scheduler 212 specifying the priority of instructionqueues 204-210 to optimize instruction execution in real time.

In some cases if the instruction scheduler 212 is able to select enoughready instructions from a higher priority queue or higher priorityqueues, then it may not even attempt to select instructions from a lowerpriority queue. For example, where there are four pipelines in theassociated functional unit 110 as shown in FIG. 2 and the instructionscheduler 212 is able to obtain four ready instructions from the highestpriority queue (e.g. instruction queue 0) and the second highestpriority queue (e.g. instruction queue 1) collectively then theinstruction scheduler may not even attempt to select ready instructionsfrom the two lower priority instruction queues (e.g. instruction queue 2and instruction queue 3).

The instruction scheduler 212 is typically configured to select theoldest ready instructions from any particular instruction queue 204-210,however, in some cases the instruction scheduler 212 may be configuredto select instructions from a particular instruction queue using othercriteria. Selecting instructions using other criteria may, however, slowdown the instruction scheduler 212 (and exceed the single clock cycletime period) if a complex set of rules and exceptions are used. In somecases the instruction scheduler 212 may select instructions fromdifferent instruction queues using different criteria (i.e. it mayselect the oldest ready instructions from one queue and use othercriteria to select instructions from another queue), however, this mayincrease the complexity of the instruction scheduler 212 and/or make itslower.

In some cases, the number of instructions that may be selected from aparticular instruction queue in a clock cycle may be limited. This maybe done (a) because there are minimal performance benefits forprioritizing more than one instructions of a certain type; and/or (b) topreempt the possibility of one or more low priority queues being starved(e.g. instructions in a particular queue are never issued to thefunctional unit 110). For example, where the reservation station 108 hasan instruction queue dedicated to control transfer instructions, theinstruction scheduler 212 may be configured to select a maximum of oneready instruction from the control transfer instruction queue in anyclock cycle. This is because it has been identified that since a secondcontrol transfer instruction is likely to be much younger than the firstcontrol transfer instruction the performance benefits for prioritizingmore than one control transfer instruction is not large compared toissuing more older instructions from another instruction queue.

When selecting instructions to issue to the functional unit, theinstruction scheduler 212 assesses a plurality of instructions todetermine if they are eligible for issue. Generally an instruction iseligible for issue if the instruction is ready and has not been issued.Determining whether an instruction is ready and has not been issued maycomprise looking at a ready column and an issue column (as described inreference to FIG. 3) of the instruction queue. However, the instructionscheduler 212 can only typically assess a maximum number ofinstructions, W_(max), per clock cycle depending on the processor 100clock frequency.

In some cases, to ensure that the instruction scheduler can assessinstructions from all instruction queues 204-210 in one clock cycle, thenumber of instructions that can be assessed in any given instructionqueue is limited to a maximum number W_(i) where i is the instructionqueue number. For example, a maximum of W₀ instructions may be assessedin the first instruction queue 204 (instruction queue 0), and a maximumof W₁ instructions may be assessed in the second instruction queue 206(instruction queue 1). The limits imposed on the instruction queues204-210 may be the same or different.

Irrespective of whether the maximums are the same or different acrossinstruction queues, the sum of the maximums (W_(i)) cannot exceed themaximum number of instructions (W_(max)) that can be assessed by theinstruction scheduler 212 in any one cycle. This is represented byequation (1) shown below where X is the number of instruction queues.

$\begin{matrix}{{\sum\limits_{i = 0}^{i = {X - 1}}W_{i}} \leq W_{{ma}\; x}} & (1)\end{matrix}$

Where there is a limit (W_(i)) on the number of instructions that can beassessed in a particular instruction queue 204-210, when the instructionscheduler 212 is selecting ready instructions from the particularinstruction queue 204-210 the instruction scheduler 212 may stopassessing instructions in the particular instruction queue when theinstruction scheduler 212 has assessed the maximum number ofinstructions (W_(i)) for that particular instruction queue. Once itstops assessing instructions in a particular queue it may move to thenext highest priority instruction queue to look for instructions toissue to the functional unit 110.

A method for selecting instructions from the plurality of instructionqueues that may be implemented by the instruction scheduler 212 isdescribed with reference to FIG. 4.

Reference is now made to FIG. 3 which illustrates an example instructionqueue 204 of FIG. 2. The instruction queue 204 may be configured as atable with a number of rows 302-332 and columns 334-338. Each row orentry 302-332 may hold information related to a single instruction. Insome cases the instructions are stored in the instruction queue 204 inthe order in which they are received. Since the instruction classifier202 typically receives and processes the instructions from the decodeand renaming stage 104 in program order, the instructions may bereceived and stored by the instruction queue 204 in program order.Accordingly, older instructions will be inserted before youngerinstructions. In the example shown in FIG. 3, instruction 0 is theoldest instruction and instruction 15 is the youngest instruction.

The columns 334-338 are used to store specific information for eachinstruction. In the example shown in FIG. 3 there are three columns: aninstruction column, a ready column and an issued column, however, inother examples there may be more, fewer, or different columns. Forexample, in some embodiments (e.g. where instructions are removed fromthe instruction queue as soon as they have been issued), the instructionqueues may not comprise an issued column.

The instruction column 334 is used to hold information describing theinstruction which is used by the functional unit 110 (e.g. pipeline) toexecute the instruction. This information may comprise the instructioncodes and input and output operands for the instruction. The instructioncolumn 334 may be implemented in any suitable manner.

The ready column 336 is used to indicate whether the instruction isready to be issued to the functional unit 110. As described above, aninstruction is said to be ready for issue to the functional unit 110when all of its input operands are available. The ready column 336 maybe implemented in any suitable manner. For example, in some cases theready column 336 may be implemented as a single bit and in other casesthe ready column 336 may be implemented as a Boolean value. When aninstruction is initially received from the instruction classifier 202its operand values are not typically available. Accordingly the readycolumn may initially be set (e.g. to “0” or “FALSE”) to indicate thatthe instruction is not ready. When the input operands subsequentlybecome available the ready column may be set (e.g. to “1” or “TRUE”) toindicate that the instruction is ready to be issued to the functionalunit 110.

The issued column 338 is used to indicate whether the instruction hasbeen issued to the functional unit 110. The issued column 338 may beimplemented in any suitable manner. For example, in some cases theissued column 338 may be implemented as a single bit and in other casesthe issued column 338 may be implemented as a Boolean value. When aninstruction is initially received from the instruction classifier 202the issued column may be set (e.g. to “0” or FALSE) to indicate that theinstruction has not been issued to the functional unit 110. Once theinstruction has been issued the issued column 338 may be set (e.g. to“1” or “TRUE”) to indicate that the instruction has been issued to thefunctional unit 110.

An instruction may remain in the instruction queue 204 until (i) it hasbeen issued; or (ii) until it has been issued and subsequentlyoverwritten by another instruction.

An instruction is typically only selected for issue to the functionalunit 110 if the instruction is ready and it has not been issued to thefunctional unit 110.

Where the number of instructions, W₀, in the instruction queue 204(instruction queue 0) that may be assessed by the instruction scheduler212 in a clock cycle is limited, a window may be established to definethe W₀ instructions to be assessed. The window may be defined by a headpointer which points to the head instruction (also referred to as thestart instruction) and a tail pointer which points to the tailinstruction (also referred to as the end instruction). In some cases thehead instruction may be the oldest instruction in the instruction queue204 that has not been issued, and the tail instruction may be the(W₀−1)^(th) instruction in the instruction queue 204 immediatelyfollowing the head instruction.

A first example is used to illustrate this concept. In particular, if amaximum of four instructions can be assessed in the instruction queue204 of FIG. 3 (e.g. W₀=4), the head pointer 340 of the window may pointto instruction 2 (head instruction) since this is the oldest non-issuedinstruction in the instruction queue 204, and the tail pointer 342 ofthe window may point to instruction 5 (tail instruction) since it is thethird (e.g. W₀−1=4−1=3) instruction immediately following the headinstruction. In this example, when the instruction scheduler 212attempts to select instructions from this instruction queue 204 (e.g.instruction queue 0) the instruction scheduler 212 assesses instructions2 to 5 one at a time until: (a) it has selected enough readyinstructions to fill the pipelines (e.g. one per pipeline), (b) it hasselected the maximum number of instructions from this queue (if amaximum has been assigned to this instruction queue), or (c) it hasassessed instruction 5 (e.g. it has assessed the maximum number ofinstruction for this instruction queue). Once one of these conditions ismet the instruction scheduler 212 stops searching the instruction queue204 for instructions to issue. If condition (a) is met the instructionscheduler 212 may issue the selected instructions to the functional unit110. If, however, condition (b) or (c) is met, the instruction scheduler212 may move to the next lower priority instruction queue (if there is alower priority instruction queue) to continue looking for instructionsto issue to the functional unit 110.

In other cases the head instruction of the window may be the oldestready non-issued instruction in the instruction queue 204, and the tailinstruction of the window may be the (W_(i)−1)^(th) instruction in theinstruction queue immediately following the head instruction. Thesecases may require more logic than the cases where the head instructionof the window is the oldest non-issued instruction in the instructionqueue 204, but it may result in a more efficient use of the instructionscheduler's 212 resources since the instruction scheduler 212 does notwaste time assessing older instructions that are not ready for issue.

A second example is used to illustrate this concept. In particular, if amaximum of seven instructions can be assessed in the instruction queue204 of FIG. 3 (e.g. W₀=7), the head pointer 344 of the window may pointto instruction 3 (head instruction) since this is the oldest readynon-issued instruction in the instruction queue 204, and the tailpointer 346 of the window may point to instruction 9 (tail instruction)since it is the sixth (e.g. W₀−1=7−1=6) instruction in the instructionqueue 204 immediately following the head instruction. In this example,when the instruction scheduler 212 attempts to select instructions fromthis instruction queue 204 (e.g. instruction queue 0) the instructionscheduler 212 assesses instructions 3 to 9 one at a time until one ofthe following conditions is met: (a) it has selected enough readyinstructions to fill the pipelines (e.g. one per pipeline), (b) it hasselected the maximum number of instructions from this queue (if amaximum has been assigned to this instruction queue), or (c) it hasassessed instruction 9 (e.g. it has assessed the maximum number ofinstructions for this queue). Once one of these conditions is met theinstruction scheduler 212 stops searching the instruction queue forinstructions to issue. If condition (a) is met the instruction scheduler212 may issue the selected instructions to the functional unit 110. If,however, conditions (b) or (c) are met, the instruction scheduler 212may move to the next lower priority instruction queue (if there is alower priority instruction queue) to look for additional instructions toissue to the functional unit 110.

Reference is now made to FIG. 4 which illustrates a method 400, whichmay be performed by the instruction scheduler 212 of FIG. 2, forselecting instructions from the instruction queues 204-210 to issue to afunctional unit 110. In the method 400 of FIG. 4 the instructionscheduler 212 searches the instruction queues 204-210 in order ofpriority for ready instructions to issue to the functional unit 110.

The instruction scheduler 212 may stop searching a particular queue ifany one of the following conditions are met: (a) the maximum number ofinstructions have been selected (e.g. the number of selectedinstructions is equal to the number of pipelines in the functionalunit); (b) the maximum number of instructions for the particular queuehave been selected (e.g. an instruction queue may have a limit on thenumber of instructions that can be selected from that instructionqueue); and (c) the maximum number of instructions for the particularqueue have been assessed (e.g. an instruction queue may have a limit onthe number of instructions in that instruction queue that can beassessed in a clock cycle). If condition (a) is met the instructionscheduler 212 may issue the selected instructions to the functional unit110. If, however, condition (b) or (c) is met, the instruction scheduler212 may proceed to the next highest priority instruction queue to searchfor ready instructions to issue.

Condition (b) may only apply if a limit on the number of selectedinstructions has been imposed on the particular queue being searched.Similarly, condition (c) may only apply if a limit on the number ofinstructions that may be assessed has been imposed on the particularqueue being searched.

This is described in more detail below in reference to steps 402 to 426.

At step 402, the instruction scheduler 212 accesses the instructionqueue with the highest priority. Using FIG. 2 as an example, ifinstruction queues 204-210 are prioritized in the following order fromhighest priority to lowest priority—instruction queue 0, instructionqueue 1, instruction queue 2 and instruction queue 3—the instructionscheduler 212 accesses instruction queue 0. Once the instructionscheduler 212 accesses the instruction queue with the highest prioritythe method 400 proceeds to step 404.

At step 404, the instruction scheduler 212 assesses the start or headinstruction to determine if it should be selected for issue to thefunctional unit 110. As described above in reference to FIG. 3, in somecases the head instruction may be the oldest non-issued instruction inthe instruction queue (e.g. instruction 2 of FIG. 3). In other cases thehead instruction may be the oldest ready non-issued instruction in theinstruction queue (e.g. instruction 3 of FIG. 3). Once the instructionscheduler has assessed the head instruction the method 400 proceeds tostep 406.

At step 406 the instruction scheduler 212 determines whether theinstruction assessed in step 404 is ready (e.g. it is ready to be issuedto the functional unit 110 for execution) and not issued (e.g. it hasnot already been issued to the functional unit 110 for execution). Ifthe instruction is ready and not issued then the method proceeds to step408. If, however, the instruction is not ready or already issued thenthe method proceeds to step 420.

At step 408, the instruction scheduler 212 selects the instructionassessed in step 404 for issue to the functional unit 110 and updates aninstruction queue counter. The instruction queue counter keeps track ofthe number of instructions that have been selected from the instructionqueue currently being accessed (the “current instruction queue”). Insome cases, updating the instruction queue counter may compriseincrementing the instruction queue counter. Once the instruction queuecounter is updated the method 400 proceeds to step 410.

At step 410, the instruction scheduler 212 updates a selectedinstruction counter. The selected instruction counter keeps track of thenumber of instructions that have been selected for issue from allinstruction queues. In some cases, updating the selected instructioncounter comprises incrementing the selected instruction counter. Oncethe selected instruction counter is updated the method 400 proceeds tostep 412.

At step 412, the instruction scheduler 212 determines whether a maximumnumber of instructions have been selected from all the instructionqueues. The maximum number of instructions that can be selected in anyclock cycle is typically equal to the number of pipelines (e.g.pipelines 214-220) in the associated functional unit 110. This isbecause only one instruction can be issued to each pipeline (e.g.pipelines 214-220) in a clock cycle. Accordingly, if the functional unit110 has four pipelines 214-220 as shown in FIG. 2 the maximum number ofinstructions may be equal to four.

Determining whether the maximum number of instructions has been selectedmay comprise comparing the selected instruction counter to the number ofpipelines. If the selected instruction counter is equal to the number ofpipelines then the instruction scheduler 212 stops looking for moreinstructions and issues the selected instructions to the pipelines. If,however, the selected instruction counter is less than the number ofpipelines than the instruction scheduler 212 may continue to look forinstructions to issue to the functional unit 110.

If it is determined that the maximum number of instructions has beenselected then the method 400 proceeds to step 414. If, however, it isdetermined that the maximum number of instructions has not been selectedthen the method 400 proceeds to step 418.

At step 414, the selected instructions are issued to the functional unit110 for execution. In some cases the instruction scheduler 212 attemptsto issue one selected instruction to each of the pipelines 214-220 ofthe functional unit 110. For example, if there are four pipelines asshown in FIG. 2 and four instructions were selected then one selectedinstruction is issued to each pipeline. If, however, there are fourpipelines and only three instructions were selected then one selectedinstruction is issued to each of three of the pipelines.

It does not generally matter which instruction is issued to whichpipeline. However, in some cases it may be beneficial to issueinstructions that require additional or special resources to a specificpipeline or set of pipelines.

For example, in some cases it may be beneficial to issue controltransfer instructions, such as branch instructions, to a specificpipeline or set of pipelines to reduce the cost of book-keeping andsynchronization between the available pipelines. In particular, controltransfer instructions may be issued to a specific pipeline so as tominimize and/or eliminate the book-keeping and synchronization requiredbetween the pipelines to track the ‘squashing’ of control transferinstructions (in case their input operands are not actually ready atexecution time); and prevent younger control transfer instructions fromovertaking older control transfer instructions.

At step 416, the selected instruction counter is reset (e.g. zeroed).Once the instruction counter has been reset, the method 400 ends.

At step 418, after an instruction has been selected for issue and it hasbeen determined that the maximum number of instructions from allinstruction queues has not yet been selected, the instruction scheduler212 determines whether the maximum number of instructions has beenselected from the instruction queue currently being accessed.

Determining whether the maximum number of instructions for theinstruction queue currently being accessed have been selected maycomprise comparing the instruction queue counter to the maximum numberof instructions that may be selected from the current instruction queue.If the instruction queue counter is equal to the maximum number ofinstructions then the instruction scheduler 212 may stop looking formore instructions from this instruction queue. If, however, theinstruction queue counter is less than the maximum number ofinstructions the instruction scheduler 212 may continue to look forinstructions from this instruction queue to issue to the functional unit110.

If it is determined that the maximum number of instructions has beenselected from this instruction queue then the method 400 proceeds tostep 424 where the instruction scheduler 212 may continue to look forinstructions from another instruction queue. If, however, it isdetermined that the maximum number of instructions has not been selectedfrom this instruction queue then the method 400 proceeds to step 420where the instruction scheduler 212 may continue to search forinstructions from the current instruction queue.

Step 418 is optional. In particular, it only applies to an instructionqueue where the number of instructions that may be selected from thatinstruction queue in a particular clock cycle has been limited. If thenumber of instructions that may be selected from the current instructionqueue has not been limited then step 418 may not be executed and themethod 400 may proceed directly to step 420.

At step 420, after an instruction has been selected for issue and it hasbeen determined that the maximum number of instructions from allinstructions has not been selected, and the maximum number ofinstructions from the current instruction queue has not been selected;or after an instruction has been assessed, but not selected; theinstruction scheduler 212 determines whether the most recently assessedinstruction is the tail instruction. As described above, the tailinstruction is the (W_(i)−1)^(th) instruction following the headinstruction where W_(i) is the maximum number of instructions that canbe assessed in the i^(th) instruction queue in a clock cycle. If themost recently assessed instruction is the tail instruction then themaximum number of instructions that may be assessed in the currentinstruction queue has been reached.

If it is determined that the maximum number of instructions has beenassessed then the method 400 proceeds to step 424 where the instructionscheduler 212 may continue to search for instructions from anotherinstruction queue. If, however, it is determined that the maximum numberof instructions has not been assessed then the method 400 proceeds tostep 422 where the instruction scheduler 212 continues to look forinstructions from this instruction queue.

Step 420 is optional. In particular, it only applies to thoseinstruction queues where a limit has been imposed on the number ofinstructions that may be assessed in a clock cycle. If the number ofinstructions that may be assessed in the current instruction queue hasnot been limited then step 420 may not be executed and the method mayproceed directly to step 422.

At step 422, the next instruction in the current instruction queue isassessed. In some cases the next instruction is the instruction in thecurrent instruction queue immediately following the most recentlyassessed instruction. Once the next instruction has been assessed themethod proceeds back to step 406.

At step 424, after (a) an instruction has been selected for issue and ithas been determined that the maximum number of instructions from allinstruction queues has not been selected, but the maximum number ofinstructions for the current instruction queue has been selected; or (b)the maximum number of instructions for the current instruction queuehave been assessed, the instruction scheduler 212 determines whetherthere are any instruction queues lower in priority than the currentinstruction queue.

If there is at least one instruction queue lower in priority than thecurrent instruction queue, the method 400 proceeds to step 426 where theinstruction scheduler 212 looks for instructions in the next highestpriority instruction queue. If, however, there are no more instructionqueues lower in priority than the current instruction queue, the method400 proceeds to step 414 where the selected instructions are issued tothe functional unit 110.

At step 426, the instruction scheduler 212 accesses the next highestpriority instruction queue with respect to the current instructionqueue. Once the instruction scheduler 212 has accessed the next highestpriority instruction queue, the method proceeds to step 404 where theinstructions in the instruction queue are assessed to determine if theyare suitable for issue to the functional unit 110.

Method 400 may be repeated each clock cycle.

While method 400 describes that the instruction scheduler 212 accessesthe queues sequentially based on priority and assesses the instructionsin a particular queue sequentially, in other examples the instructionscheduler may comprise hardware that: (i) speeds up the assessment ofinstructions in a particular queue by allowing the instruction scheduler212 to determine information about the next instruction in the queuewhile assessing the current instruction; (ii) allows it to assessmultiple instructions in a queue in parallel; and/or (ii) allows it toaccess and assess instructions in multiple queues in parallel.

In other examples, the instruction scheduler 212 may use standardselection logic (e.g. as described in ‘Delay Evaluation of Issue Queuein Superscalar Processors with Banking Tag RAM and Correct Critical PathIdentification’ by Yamaguchi et al, published in IEICE TRANS. INF. &SYST., VOL. E95-D, NO. 9 SEPTEMBER 2012) to select instructions fromwithin each queue such that a total of N instructions are selected everycycle (one for each pipeline). While the examples described above relateto single-threaded out-of-order processors, the methods and principlesdescribed herein may equally be applied to multi-threaded out-of-orderprocessors.

Reference is now made to FIG. 5 which illustrates a schematic of amulti-threaded out-of order processor 500. The processor 500 comprisestwo threads 502 and 504 which will be referred to herein as thread 0 andthread 1 respectively. Each thread 502 and 504 comprises a fetch stage506 or 508, a decode and renaming stage 510 or 512, a re-order buffer514 or 516, a commit stage 518 or 520, and a branch predictor 522 or 524as described above in reference to FIG. 1. The threads 502 and 504 sharethe reservation stations 526 a, 526 b and 526 c, and functional units528 a, 528 b and 528 c as described above in reference to FIG. 1. Alsoas described above, each functional unit 528 a, 528 b and 528 c maycomprise one or more pipelines (not shown) for executing theinstructions issued to the functional unit 528 a, 528 b or 528 c.

Each decode and renaming stage 510 and 512 analyzes each fetchedinstruction and then dispatches the instruction to the appropriatereservation station 526 a, 526 b or 526 c. In accordance with theprinciples and methods described above in reference to FIGS. 2 to 4,each reservation 510 and 512 sorts the received instructions into aplurality of categories based on the type of instruction; stores eachcategory of instructions into a separate instruction queue; selectsinstructions from the instruction queues in a manner that prioritizescertain types of instructions over other instructions even if they areyounger than the other instructions; and issues the selectedinstructions to the associated functional unit 528 a, 528 b or 528 c.

In some cases, each thread may have a separate set of categories andassociated instruction queues, although the sets of categories may bethe same for each thread (e.g. a category for control transferinstructions, a category for address generation instructions, a categoryfor flag setting instructions and a category for general purposeinstructions). An example of these different queues are shown in theupper schematic diagram 601 in FIG. 6. In this example there are twothreads, thread 0 and thread 1 and three queues 602 a-606 a, 602 b-606 bfor each thread, one for branch instructions (queues 602 a, 602 b), onefor address generation instructions (queues 604 a, 604 b) and one forgeneral purpose (GP) instructions (queues 606 a, 606 b). In these casesthere may be one instruction scheduler for each thread that selectsinstructions from its associated instruction queues in accordance withthe principles described above in reference to FIGS. 2 to 4. In thesecases the reservation stations 526 a, 526 b and 526 c may also comprisean arbitration unit (not shown) that selects instructions from the twothreads in accordance with a set of rules. For example, the arbitrationunit may select instructions from the two threads based on howspeculative each thread's instructions are as described in co-pending UKpatent application “ALLOCATING RESOURCES TO THREADS BASED ON SPECULATIONMETRIC” Jackson et al. filed on 21 Jan. 2013.

In an example scenario for a multi-threaded processor (or processorcore), there may be two threads (thread0 and thread1) and 4 queues forthread0 and 4 queues for thread1 (e.g. one queue for each of fourcategories: control transfer instructions, address generationinstructions, flag setting instructions and general purposeinstructions). In this example there may be 4 pipelines which are sharedbetween thread0 and thread1 and so generally 2 pipelines are availablefor thread0 and 2 pipelines are available for thread1.

Starting with thread0, one instruction from each queue may be issuedinto each of the 2 pipelines available for thread0 or if there areadditional ready instructions in a higher priority queue, there may bemore than one instruction issued from that queue, resulting in noinstructions being issued from one or more lowest priority queues (e.g.1 instructions from the highest priority queue, 1 instruction from thesecond highest priority queue and no instructions from the remaining twolower priority queues). Similarly, instructions for thread1 are issuedfrom one or more of the 4 queues for thread1 based on the relativepriority of these queues. If either thread does not have sufficientready instructions, the other thread takes up its pipelines (e.g. ifthread0 only has 1 ready instructions, thread1 may issue 3instructions).

Although this is described in terms of issuing ready instructions fromthe separate queues in order of priority, this is conceptuallyequivalent to combining all eligible instructions for a thread into onequeue, with the instructions being arranged in order of priority oftheir category and issuing from that queue. This process is done foreach thread and is represented graphically in the lower schematicdiagram 610 in FIG. 6. In this lower diagram, each consolidated queue611 a, 611 b for each thread comprises ready instructions from each ofthe separate queues 602 a-606 a, 602 b-606 b (in the upper diagram 601)with the instructions being ordered based on the relative priority ofthe separate queues, e.g. ready branch instructions 612 a, 612 b (fromqueues 602 a, 602 b) followed by ready address generation instructions614 a, 614 b (from queues 604 a, 604 b) followed by ready generalpurpose instructions 616 a, 616 b (from queues 606 a, 606 b).

Generally it does not matter which instruction is issued to whichpipeline. However, in some cases instructions from a particular threadmay be issued to the pipelines in a predetermined order. For example, ifthere are four pipelines numbered 0 to 3 any instructions selected fromthe first thread (e.g. thread 0) may be first issued to the firstpipeline (e.g. pipeline 0) and then to subsequent pipelines (e.g.pipeline 1) in ascending order; and any instructions selected from thesecond thread (e.g. thread 1) may be first issued to the last pipeline(e.g. pipeline 3) and then to subsequent pipelines (e.g. pipeline 2) indescending order.

Since each pipeline in the functional units 528 a, 528 b and 528 c canend up with instructions from more than one thread, when the instructionscheduler issues instructions to the pipelines it may also provide thepipelines with an indication of which thread the instruction isassociated with. Then, if an incorrectly predicted instruction occursrequiring a rewind of the pipelines, the pipelines can rewind or flushonly those instructions that are associated with the thread with theincorrectly predicted instruction instead of rewinding all instructions.In some cases the incorrectly predicted instruction may be allowed tocontinue in the pipeline until all of the instructions up until theincorrectly predicted instruction have been executed. At that point allinstructions in the pipelines and the reservation station 526 a, 526 bor 526 c from the particular thread have occurred after the incorrectlypredicted instruction and need to be rewound or flushed.

In a variation of the methods described above and for multi-threadedprocessors (or processor cores), instructions of the same category butfor different threads may combined to form a single queue for thecategory by interleaving the instructions from the separate queues foreach category. This may be described with reference to FIG. 7. In theupper diagram 601 of FIG. 6 (described above) there is one queue foreach thread and for each category; however, as shown in the upperdiagram 701 in FIG. 7, there is one queue for all threads for eachcategory: one queue for branch instructions 602 c, one queue for addressgeneration instructions 604 c and one queue for general purposeinstructions 606 c. As indicated by the striping on each queue 602 c,604 c, 606 c in FIG. 7, these queues comprise interleaved instructionsfrom the two threads, thread0 and thread1.

By combining the queues for the threads in this way (i.e. byinterleaving), it is possible to prioritize instructions of a particularcategory (from both thread0 and thread1) over all other instructionsfrom either thread. When selecting instructions to schedule, theinstruction scheduler assesses the interleaved queues in order ofdecreasing priority and selects instructions for issue using the methodsdescribed above.

This can be further described using the same example scenario as abovewith 2 threads (thread0 and thread1) 4 pipelines, 4 queues for thread 0and 4 queues for thread 1. The 4 pipelines are shared between thethread0 and thread1, but there is no reservation of 2 pipelines forthread0 and 2 for thread1.

In determining which instructions to issue, the interleaved queues areassessed in order of decreasing priority, so, for example, if 1 branchinstructions from thread0 and 3 branch instructions from thread1 wereready to be issued, then all 4 branch instructions would be issued thatcycle. In contrast, without interleaving, 1 branch instruction would beissued from thread0, then an address generation instruction from thread0and 2 branch instructions from thread1.

By interleaving instructions from each thread, both threads have equalpriority and instructions are issued based on the relative priority ofthe queues (or the categories which define the queues).

In a similar manner to the earlier example, although this is describedin terms of issuing ready instructions from the separate queues (602 c,604 c, 606 c) in order of priority, this is conceptually equivalent tocombining all eligible instructions for all threads into one queue, withthe instructions being arranged in order of priority of their categoryand being interleaved (for the different threads within each category)and issuing from that queue. This process is represented graphically inthe lower schematic diagram 710 in FIG. 7. In this lower diagram, eachconsolidated queue comprises ready instructions from each of theseparate queues 602 c-606 c (in the upper diagram 701) with theinstructions being ordered based on the relative priority of theseparate queues, e.g. interleaved ready branch instructions 702 (fromqueue 602 c) followed by interleaved ready address generationinstructions 704 (from queue 604 c) followed by interleaved readygeneral purpose instructions 706 (from queue 606 c).

Using the interleaving of instructions from each thread, as describedabove, resources can be dedicated and instructions prioritized based onthe instruction type (e.g. based on the category); whilst still givingeach thread equal priority within a category. Otherwise, if interleavingis not used, less critical instructions from one thread may be issuedahead of higher priority instructions from another thread, because theinstruction scheduler attempts to share resources equally betweenthreads unless there are insufficient ready instructions. Theinterleaving, as described with reference to FIGS. 6 and 7 may thereforeresult in improved performance of the processor.

It will be appreciated that there may be no real interleaved queue inhardware when implementing the methods described above; however, theinterleaved queues described conceptually the way that readyinstructions are selected for issue by the instruction scheduler.

Those skilled in the art will realize that storage devices utilized tostore program instructions can be distributed across a network. Forexample, a remote computer may store an example of the process describedas software. A local or terminal computer may access the remote computerand download a part or all of the software to run the program.Alternatively, the local computer may download pieces of the software asneeded, or execute some software instructions at the local terminal andsome at the remote computer (or computer network). Those skilled in theart will also realize that by utilizing conventional techniques known tothose skilled in the art that all, or a portion of the softwareinstructions may be carried out by a dedicated circuit, such as a DSP,programmable logic array, or the like.

Memories storing machine executable data for use in implementingdisclosed aspects can be non-transitory media. Non-transitory media canbe volatile or non-volatile. Examples of volatile non-transitory mediainclude semiconductor-based memory, such as SRAM or DRAM. Examples oftechnologies that can be used to implement non-volatile memory includeoptical and magnetic memory technologies, flash memory, phase changememory, resistive RAM.

Any range or device value given herein may be extended or alteredwithout losing the effect sought, as will be apparent to the skilledperson.

It will be understood that the benefits and advantages described abovemay relate to one embodiment or may relate to several embodiments. Theembodiments are not limited to those that solve any or all of the statedproblems or those that have any or all of the stated benefits andadvantages.

Any reference to ‘an’ item refers to one or more of those items. Theterm ‘comprising’ is used herein to mean including the method blocks orelements identified, but that such blocks or elements do not comprise anexclusive list and an apparatus may contain additional blocks orelements and a method may contain additional operations or elements.Furthermore, the blocks, elements and operations are themselves notimpliedly closed.

A particular reference to “logic” refers to structure that performs afunction or functions. An example of logic includes circuitry that isarranged to perform those function(s). For example, such circuitry mayinclude transistors and/or other hardware elements available in amanufacturing process. Such transistors and/or other elements may beused to form circuitry or structures that implement and/or containmemory, such as registers, flip flops, or latches, logical operators,such as Boolean operations, mathematical operators, such as adders,multipliers, or shifters, and interconnect, by way of example. Suchelements may be provided as custom circuits or standard cell libraries,macros, or at other levels of abstraction. Such elements may beinterconnected in a specific arrangement. Logic may include circuitrythat is fixed function and circuitry can be programmed to perform afunction or functions; such programming may be provided from a firmwareor software update or control mechanism. Logic identified to perform onefunction may also include logic that implements a constituent functionor sub-process. In an example, hardware logic has circuitry thatimplements a fixed function operation, or operations, state machine orprocess.

The steps of the methods described herein may be carried out in anysuitable order, or simultaneously where appropriate. The arrows betweenboxes in the figures show one example sequence of method steps but arenot intended to exclude other sequences or the performance of multiplesteps in parallel. Additionally, individual blocks may be deleted fromany of the methods without departing from the spirit and scope of thesubject matter described herein. Aspects of any of the examplesdescribed above may be combined with aspects of any of the otherexamples described to form further examples without losing the effectsought. Where elements of the figures are shown connected by arrows, itwill be appreciated that these arrows show just one example flow ofcommunications (including data and control messages) between elements.The flow between elements may be in either direction or in bothdirections.

It will be understood that the above description of a preferredembodiment is given by way of example only and that variousmodifications may be made by those skilled in the art. Although variousembodiments have been described above with a certain degree ofparticularity, or with reference to one or more individual embodiments,those skilled in the art could make numerous alterations to thedisclosed embodiments without departing from the spirit or scope of thisinvention.

1. A method of selecting instructions to issue to a functional unit inan out-of-order processor, the method comprising: receiving a pluralityof instructions at a reservation station, each instruction being one ofa plurality of types of instructions and being classified into one of aplurality of categories based on the type of instruction; storing, atthe reservation station, each instruction in one of a plurality ofinstruction queues, each instruction queue being associated with one ofthe plurality of categories, each instruction queue being assigned apriority and a maximum number of instructions assessable in a clockcycle; and selecting, at the reservation station, instructions from oneor more of the instruction queues to issue to the functional unit inorder of the assigned priorities so that instructions are selected basedon a relative priority of the plurality of types of instructions;wherein the selecting instructions from a particular instruction queuecomprises assessing, in a clock cycle, each of one or more instructionsin that particular instruction queue up to the maximum number ofinstructions for that particular queue to determine if that instructionis ready and has not been previously issued.
 2. The method according toclaim 1, wherein the method further comprises classifying, at one of thereservation station and a decode and renaming stage, each instructioninto one of the plurality of categories based on the type of instruction3. The method according to claim 1, wherein instructions are selectedfrom the instruction queues in order of the priorities until a maximumnumber of instructions have been selected.
 4. The method according toclaim 3, wherein the functional unit comprises one or more pipelines andthe maximum number of instructions selected is equal to the number ofpipelines.
 5. The method according to claim 1, wherein selectinginstructions from a particular instruction queue further comprisesestablishing a window covering instructions to be assessed in thatparticular queue, the window being defined by a head pointer pointing toa head instruction and a tail pointer pointing to a tail instruction. 6.The method according to claim 5, wherein the head instruction is theoldest instruction in the particular instruction queue that has not beenissued.
 7. The method according to claim 5, wherein the head instructionis the oldest instruction in the particular instruction queue that isready and has not been issued.
 8. The method according to claim 5,wherein the tail instruction is the (X−1)^(th) instruction in theparticular instruction queue following the head instruction, wherein Xis the maximum number of instructions assessable in the particularinstruction queue.
 9. The method according to claim 1, wherein selectinginstructions from a particular instruction queue comprises selecting upto a maximum number of instructions from the particular instructionqueue.
 10. The method according to claim 1, wherein the plurality ofcategories comprises a category for at least one of control transferinstructions, flag setting instructions and address generationinstructions.
 11. The method according to claim 1, wherein the pluralityof categories comprises a category for control transfer instructions, acategory for flag setting instructions and a category for addressgeneration instructions.
 12. The method according to claim 10, whereinthe instruction queue associated with the category for control transferinstructions is assigned the highest priority.
 13. The method accordingto claim 1, further comprising issuing, at the reservation station, theselected instructions to the functional unit for execution.
 14. Themethod according to claim 13, wherein the functional unit comprises oneor more pipelines and issuing the selected instruction to the functionalunit comprises issuing each selected instruction to one of thepipelines.
 15. The method according to claim 1, wherein the out-of-orderprocessor is a multi-threaded processor and instructions are selectedfrom the instruction queues based on a relative priority of theplurality of types of instructions and within a type of instruction,instructions are selected from all threads in the multi-threadedprocessor.
 16. A reservation station to select instructions to issue toa functional unit in an out-of-order processor, the reservation stationcomprising: an instruction classifier configured to receive a pluralityof instructions, each instruction being one of a plurality ofinstruction types and configured to classify into one of a plurality ofcategories based on the type of instruction; a plurality of instructionqueues in communication with the instruction classifier, eachinstruction queue configured to receive instructions from theinstruction classifier that have been classified into a particularcategory and store the received instructions, each instruction queuebeing assigned a priority and a maximum number of instructions assessedin a clock cycle; and an instruction scheduler in communication witheach of the plurality of instruction queues, the instruction schedulerconfigured to select instructions from one or more of the instructionqueues to issue to the functional unit in order of the assignedpriorities so that instructions are selected based on a relativepriority of the plurality of types of instructions; wherein selectinginstructions from a particular instruction queue comprises assessing, ina clock cycle, each of one or more instructions in the particularinstruction queue up to the maximum number of instructions for thatparticular queue to determine if the instruction is ready and has notbeen previously issued.
 17. The reservation station according to claim16, wherein selecting instructions from a particular instruction queuefurther comprises establishing a window covering the instructions in aparticular queue to be assessed, the window being defined by a headpointer pointing at a head instruction and a tail pointer pointing at atail instruction.
 18. A processor comprising the reservation station ofclaim
 16. 19. A computer readable storage medium having encoded thereoncomputer readable program code for generating a processor comprising thereservation station of claim
 16. 20. A computer readable storage mediumhaving encoded thereon computer readable program code for generating aprocessor configured to perform the method of claim 1.